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This paper describes the process implemented in the 
development of ;n instrument to measure baseline levels of applied 
Total Quality Management (TQM) practices in South Dakota prior to 
the introductici and dissemination of TQM theory to the state's 
e ducat ional lea lers . 

Using the interpretation of Demings 14 points as developed by 
Bonstingl, the investigators developed a total of 115 items 
(appro x i ma t e 1 y eight per each of the 14 points) for the initial 
i t em poo 1 . 

Using a modified Q sort procedure the items were subjected to 
a panel of 15 judges ""who examined them for clarity and 
appropriateness. This process was initiated to establish content 
val idi ty . 

After the sorting process was completed, 68 items remained (^4 
per subscalc) and were subjected to pilot testing using the 
responses of 77 practicing K-12 teachers representing 12 school 
buildings in six different school districts. An analysis was 
performed to establish reliability coefficients (Cromback's 
Coefficient Alpha) for the 14 subscales and the total test. Ten of 
the subscales examined reveled Alpha coefficients >.71 (range = .71 
to .91) and a total instrument reliability of .96. It was 
determined that two of the subscales are in need of further 
development to strengthen differentiation within Mass Inspection 
(Deming point 3) and Elimination of Targets (Deming point 10). One 
of the subscales Point 4 wi 1 1 De discarded because vendor service 
and quality concerns are moot in a public enterprise that is 
subjected to the lowest bidder mentality. 
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The Development cf a Survey Instrument 
on South Dakota's School District Leadership Climate 
as Related to Deming's Fourteen Points 

INTRODUCTION . . 

During the past decade, there hos been growing concern 
regarding education in the United States. In 1983, the National 
Commission on Excellence in Education published its famous 
pessimistic report titled, A Nati on At Risk , which mandated 
intensive and massive restructuring of American education. Other 
commissions and reports followed at both state and federal 
levels. In response, many educational administrators called for 
a more systematic approach in the restructuring movement and 
suggested that Deming's Fourteen Points and Total Quality 
Management (TQM) . which radically affected leadership management 
in industry, should also b- applied in education (Rhodes, 1990; 
Kelvin, 1991; and Bonstingl , 1392). 

The Fourteen Points are a management philosophy and process 
with each point having an inherent inter-dependence upon all 
other points. An educational version of the Fourteen Points is 
as foil ows : 

1 . Constancy of purpose . 

2, Adopt the n^w philosophy. 
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3. Cease dependence on mass inspection. 

4. Cease doing business on price tag alone. 

5 . Continual improvement of process . 

6. Institute training on the job. 

7 . Institute leadership. 

8 . Drive out fear . 

9 . Break down bar ri ers ( among ) departments . 

10 . El iminate slogans, exhortations , and targets . 

11 . Zl iminate num^rrcu] quotas . 

12. Allow pride in workmanship. 

13. Institute a program of sel f -improvement . 

14. Do it. (Reilly, 1991) 

We believe that ultimately school districts in South Dakota 
and other states will be influenced by Deming's Fourteen Points 
and TQM in their restructuring efforts. This influence is 
beginning to be shown through the 1993-1994 theme for the South 
Dakota LEAD Project and in the South Dakota Modernization 
(restructuring) Project ! s seven components, which are as follows 

1. A clear vision cf how the school will function, 
and a strategic plan for bringing everyone up to 
speed . 

2. Expanded learning environments, opening the door 
- to the community for learning opportunities and - 

involving the community in school planning. 

3. Commitment to outcome based education, where 
students advance by reaching clearly defined, exit 
outcomes, not by just putting in time. There mus 
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also be a system for measuring student outcomes, 
and a policy of using student achievement as the 
basis for school planning. 

4. Use of authentic: tasks... applying subject matter 
to the real world instead of relying on textbooks 
and contrived exercises. 

5 . Commi tment to technical 1 i teracy , not onl y 
regard] ng tod *y ' s f echnol ogy , but hel pi nq stud^nt.r. 
feel comfortable enou:;::* t/> adapt :.o t cr.:->r row * r 

t echno 1 ogy . 

6. Entrepreneurial studies, helping students see ways 
of creating their own jobs in the next century. 

7 . Cooperative 1 earning , he 1 ping students 1 earn to 
work as team players. (Higbee. p7 ) 

After reviewing the literature and finding no instruments 
available to measure perceived use of TQM and the Fourteen 
Points, the investigator's elected to develop an instrument to 
measure the existing level of applied TQM practices in the 
schools and to establish this "level of application prior to any 
of the training/awareness efforts. The administration of such an 
instrument to a comprehensive sample of South Dakota's teachers 
would, we believe, establish :\ baseline for future studies. 



METHODOLOGY 

The instrument was developed by the authors using the 
foil owing procedures. 

Referen ces : After extensive loading on Deming and applied TQM , 
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the authors 1 settled on the baseline conceptualization of 
Deming's 14 Points as presented by BonstingI (1992). We 
considered this to be the single most comprehensive and accepted 
treatise on the application of Dewing f s theories to the school 
setting. A copy of BonstingI \s educational interpretation of 
Deming's Fourteen Points is found in Appendix A . 

F ormat : A pool of items was developed in statement form based o 
each of the Fourteen Points. Roth positive and negative 
statements were developed for an iritial total of 101 survey 
items. It was intended that, through the following steps in the 
process, the actual number of items be reduced from 101, to 68. 
and an eventual total of 52 items (four statements each for 
thirteen of the Fourteen Points). Formatting sources included 
organizational climate instruments such as The Prof i 1 e__of 
Organizational Practices (POP) authored by Zigarmi, Edeburn, and 
Blanchard (1982), and the Staff Develop ment a nd School Climate 
Assessment Ques t i onnai re authored by Zigarmi and Edeburn ( 1980 ). 

VALIDITY 

Co ntent Validity : A Q-Sort established the content validity. 
Four educators in higher education served as an initial jury by 
matching the randomly mixed 101-item statements to the perceived 
point it represented. The jurors were also given BonstingI f s 
educational interpretation of the Fourteen Points to use as a 
reference for each of the points. The results of their choices 
were analyzed and those items having the most consensus were hep 
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and the others discarded. Some of the remaining statements were 
reworded in an effort to gain more clarity and content integrity 

A new total of sixty-eight items remained. The final 
instrument is intended to be used with administrators in 
education, therefore thirteen individuals with educational 
administration experience were asked to serve as the juiy ^or th 
second phase of this process. They included three elementary 
principals, two secondary pri-.cip.nls. two supt-rintHiidents , four 
university professors, and two deans (one from education and cne 
from* library science). All thirteen agreed to participate and 
matched the remaining randomly mixed statements to each of the 
Fourteen Points. Again, each was supplied with Bonstingl's 
educational interpretation of the Fourteen Points and directions 
for completing the process. 

A Q-Sort was again done to determine consistency between 
each of the statements and its perceived point. The data 
determined in this analysis r-v^aled that there was definite 
consensus regarding the matching of points for the majority of: 
the statements except for the last point. The fourteenth point 
is somewhat of a synthesis of the preceding thirteen points whi-: 
directs the action to carry out this point. At that time, it wa 
determined that only the first thirteen points and their 
statements would be used in the survey. The remaining statement 
were reviewed and some were minimally revised, again to 
strengthen clarity (i.e., consistent use of the terms "school" 
and "school district") and the connection to its perceived point 
Several of the jurors had suggested revisions and these were ale 
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considered in the revision process. Due to the consensus 
determined by the jurors, it is felt that content validity has 
been established. 

RELIABILITY 

Using the 68 items, a pilot instrument was designed using a 
six point Likert Scale application (Strongly Disagree, Disagree, 
Partly Disagree, Partly Agree, Agree., and Strongly Agree). Th* 
instrument, tentatively titled th<-> Total Quality Management, in 
Education Opinion Survey, wa,; piloted with a total of eighty-si:-: 
respondents. The respondents included 38 elementary teachers, 2 
secondary teachers, 4 adminis tL ators , and 22 other educators 
(e.g., counselors, special education'. The application of this 
process was used to address internal consistency and reliability 

The purpose of establishing reliability is to reduce 
measurement error on a specific instrument. In the application 
of internal consistency and reliability, an instrument is 
considered reliable if the items are interrelated. The process 
used in this study roughly parallelled the Situational Leadershi 
Follow- Up Su rvey: A Pilots t udy . i n._ E ducat i on a 1__ 7idn;,i.ni s tr a t i on , 
authored by Edeburn and Li 1 1 g r e n (19 90), and the Kn o w 1 e d o e S v s t e :t 
Invent ory: Inst rumen t _D p ye jo z. m e n t and Pi±ot_Study, authored by 
Johnson- and Edeburn (1991), studies relative to rel iabi 1 ity/i tern 
analysis procedures. Data were retrieved and subjected to 
computer analysis utilizing TESTAT, a computer program which 
employs Cronbach's Coefficient (Alpha) (Crc.ibach, 1951) to 
determine the internal consistency . rel iabi 1 ity coefficient for 
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each subscale and for the total instrument. TESTAT also provides 
the researcher with individual item analyses so that individual 
items can be examined, purged, or adjusted in the instrument 
process. The results of thes- analyses aie reported in tables 1, 
2, and 3. 

RESULTS 

The summary table of in.-a: r .. rf .ii.l.iiv -L- v , a t i o i.-;. a:.'- 
reliability coefficient:.?, .ir*- l--"^.' : in T?bl^ i. c r • * ^ i * 
subscaies analyzed, 1C reveal t-J 3co:pt a!,i c- aiph,* s.'-nr^.;, These 
included scales i, 2, 5, 6, 7. «i, 9, ] 1 12, and 13. Thf overall 
reliability score of ,9G is rvc-llenl . Three of th* subscal^r 
need further work as indicated by the lower reliability 
coefficients. The investigates have determined that the two 
subscales, Mass Inspection (subscale 3) and Numerical Quotas 
(subscale 10), are interrelated. Our immediate reaction is that 
educators, especially teachers, are generally mixed in regard to 
their feelings about testing, testing programs, and numerical 
measurement schemes of any variety, Coupled with this is the 
interpretive position taken by Demir.g i -i regard to th^se 
applications. His approach to mass inspection is nec;?ti\;o while 
his approach to systematic statistical analysis of production 
elements in the interest of i;: proving systemic functioning is 
positive. These conceptual differences coupled with educator's 
dependency on and/or rejection of standardised testing pi oc>-sst;s 
apparently "muddied the water" in these subscales. The authors 
spent a good deal of time writing, re-writ ing and revising the 
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items in anticipation of this potential confusion. We will have 
to examine the individual ii^rn responses (see tables 2 and 3) 
carefully in further efforts. 

In regard to subscale 4, End Price Tag, the r es.Harch~L s 
anticipated similar results. Whereas Deming encourages 
organisations in thr commercial sector to' use vendors who provic- 
quality materials and services regardless of price, educators as 
participants in the public sector have always been at the mercy 
of the lowest bidde. - . We believ- Mi^t t lie appaient confusion 
w it:/ t :'» e f i v e a \\ J w r s t o i t t i n .v-.!.-.:\'ai £■ -: , is d u c t o z h i s 
paradigm shift and hope to find further clarification in item 
anal ysis . 

In summary, the authors are extremely pleased with the 
overall results of the reliability analysis. Even though the 
analysis has indicated that some work is yet to be done, the 
analysis has made that job easier. 

The individual item analysis was summarized in Table 2. A/, 
can be noted in the results, all of the items (see the R-SCALE 
column) with the exception of item 57 were >,30, our pre- 
determined cut off level. What, remains to be accomplished is a 
more detailed scrutiny of the it«rin r i a tenants with score::- of < . 7 ? 
in the interest of determining whether or not clarity has been 
effected by word juxtaposi t'i cn oi sentence construction. This 
process has already been initiated. Several of these items have 
"or" choices which are, we believe, contributing to the lower 
correlation coefficient. 

Table 3 was used to display the distribution of item 
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responses by percentages. This effort was initiated to visually 
display the distributions and to determine whether or not Un- 
reversed items (see (REV) column) were appropriate, As can be 
noted, items 2, >, , Z, 16, 17, i3, ~ \ , ?7. , 51, G , and GS, arc 
reverse scored. This procedure i.v used to provide variety to 
response reactions of the pa , ti c; p..-»n ts and o approach questions 
from the negative point of view. Th»'* key to the reversal proces: 
is that SA scores of - ar- v- i v«.-l ' o a of } . scoi of! 5 

are rev'*!, sed t-'j ~ and r»n ~\ - -.■ 1 i fc .. aro pi .^a-.ed .;: ' :. 
tho'uesults both i;i regaid t» ' Ir.- 

DISCUSSION 

The need for an overall manay ox^n t and administration 
process in education has traditionally been that of scientific 
management, as espoused by Fr edi. icl: Taylor at the turn of the 
century. One example theory among many constituting ensuing 
support and further exploration of that philosophy is Max Weber 1 
Theory of Bureaucracy and th*r ma jo. constructs of hieiai. chy of 
offices, rules and regulations, specialization of tasks, 
impersonality, written record;-, etc 'Sliver, L" 7 v--*:). The 
theories that followed explained and advocated scientific 



management but did not promote overall ,-n;:ce^:; f 



c r e a c ; 



• ) 



individual in the system, iath« 
"survival of the fittest.' 1 

Schools became dependent upon score results as the primal y 
means of justification for existence - . The new justification, oi 
standard for success, is to assist the student, in learninct and 
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have every student learn to the best of that student's ability. 
This is where an overall process instead of a class/student 
comparison and assessment system is needed.. Deming's Fourteen 
Points has. the management answer I- hot advocates an individual 
student (customer) success ciiented process. 

By learning how closely aligned individual school district 
are to the Fourteen Points at any given time, districts would b 
abl e to work on the low al i pw\nLs . 7)^: survey i^puited 

up-:n in this pace:* is p.ii"t -t ".hat : . .-: r. rur:.en devel op me:; 1 . . 

'The results of this suivey and the resulting work now bein 
done to strengthen the instrument has promoted more research fo 
a more positive school district alignment to the Fourteen Point 
and TQM. Although this instrument is being developed for use i 
South Dakota school districts, we fee' that the instrument will 
have an application f o * use in oth^i states as well. 
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APPENDIX 1 

Bonstingl , 1992, p. 77-82. 



1.. . Create a. corstancy of purpose for in.pi. ovement of product an 
service. 

School must focus on helping students to maxind t.l,,--; 
own potentials through continuous improvement of teachers 1 and 
students' work together. Ma Mini .tat on of test scores and 
assessment symbol- is less i.v j t.a:;t. than th>< progress inherent 
in the continuous learning process " of each student. 

2. Adopt the new philosophy. 

School leaders must adopt and fully support the new 
philosophy of continuous improvement through greater empowerment 
of teacher-student teams. Cynical application of the new 
philosophy, with the sole intent of improving district -wide 
scores, destroys i r. t "rpfti'son.i tin.;* which., is essential to 
success . 



3. Cease dependence on mass 

Reliance on tests as 
student production is inherent 
reliable nor authentic. It i: 
assess students 1 progress if t 
productivity. Tests and other 
should be given as diagnostic* 



inspection . 

the major means of assessment of 
Ly wasteful and often neither 

too late at the end of the unit t 
he goal is to maximize their 

indi cators of student 1 earning 
a .nd y\ t-scriptive instruments 
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throughout the learning process. Learning is best shown by 
students' performance, applying infownation and skills to real- 
life challenges. Students must be taught how to assess their ow 
work and progress if they are to take ownership of their own 
educational processes, 

4 . End the practice of doing business on price tag al one . 

Build relationships of trurt and collaboration within 
the school, and between echo. 1 . ai.-" :w-:»:-n school and 11:*- 
cornrrvuni t y . Everyone's roles as supplier and customer must be 
recognized and honored. Work together whenever possible to 
maximize the potentials of students, teachers, administrators, 
and the community. 

5. Improve constantly and forever the system of production anc 
servi ce . 

School administrators must create and maintain the 
context in which teachers are empowered to make continuous 
progress in the quality of th-ir learning and other aspects of 
personal development, while they learn valuable lessons from 
( temporary ) failures. 

6. Institute programs of tiainiug. 

School leaders must institute programs of training fo 
new employees unfamiliar with the specific culture and 
expectations of the school. Effective training programs show n 
teachers how to set goals, how to teach effectively, and how to 



15 17 
BEST COPY AVAILABLE 



assess the quality of their work with students. Teachers must 
also institute programs in which students learn how to set 
learning goals, how to be more effective in their school work, 
and how to assess the quality of their own work. Teachers shoul u 
show students be attitude and actions what a good learner is all 
about (Educators learn how to be educators from the modeling they 
receive as students.) 

7. Institute leadership. 

School leadership consists of working with teachers, 
parents, students, and members of the community as coach and 
mentor so that the organizational context in which all students 1 
growth and improvement is valued and encouraged can be maximized 
by teachers and students, parents, and community members who 
support the common effort. Leading is helping, not threatening 
or punishing. 

8. Drive out fear. 

Fear is counterproductive in school as it is in the 
workp lace. Fear is destructive of the school culture and 
everything good that is intended to take place within it. 
.Institutional changes must reflect shared power, shared 
responsibilities, and shared rewards. 

9. Break down barriers between staff areas. 

Teacher and student productivity is enhanced when 
departments combine talents to create more integrated 
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opportunities for learning and discovery. Create cross- 
departmental and multi-level quality teams to break down role and 
status barriers to productivity. 

10. Eliminate slogans, exhortations, and targets for the 
workforce. 

Teachers, students, administrators, families, and 
community members may collectively arrive at slogans and 
exhortations to improve their work together, as long as power, 
responsibility, and rewards are equitably distributed. When 
educational goals are not met, fix the system instead of fixing 
blame on individuals. 



11. Eliminate numerical quotas. 

Assignments and tests that focus attention on numerical 
or letter symbols of learning and production often do not fully 
reflect the quality of student progress and performance. When 
the grade becomes the bottom-line product, short-term gains 
replace student investment in long-term learning, and this may 
prove counter-productive in the long run. 

12. Remove barriers to pride and joy in workmanship. 

Teachers and students generally want to do good work 
and feel pride in it. School; must dedicate themselves to 
removing the systemic causes of teacher and student failure 
through close collaborative efforts. 
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13. Institute a vigorous program of education and retraining. 

All of the school's people benefit from encouragement 
to enrich their education by exploring ideas and interests beyond 
the boundaries of their professional and personal worlds. 

14. Take action to accomplish the transformation. 

School personnel at all levels (including students) 
must put this new philosophy into srtion so it becomes imbedded 
into the deep structure and cultiue of the school. Teachers and 
students aione cannot put the plan into effect. Constant top- 
level dedication to full implementation must be supported by a 
critical mass of school and community people to implement the 
plan and make it stick. (Bonstingl, 1992, p. 77-82) 
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TABLE #1 

SUMMARY TABLE OF MEANS, STANDARD DEVIATIONS AND 
RELIABILITY COEFFIEIENTS (ALPHA)* 

POINTS # OF ITEMS MEANS SD ALPHA 

1. Constancy of Purpose 7 29.30 5.98 .80 

2. New Philosophy 4 17.64 3.38 .73 

3. Mass Inspection 5 15.79 2.55 .16 

4. End Price Tag 5 16.70 2.89 .26 

5. Improve Constantly 4 16.27 3.90 .81 

6. Training Programs 5 16.07 5.09 .80 

7. Institute Leadership 7 26.17 7.80 .91 

8. Drive Out Fear 6 22.34 5.10 .80 

9. Break Down Barriers 6 21.21 4.95 .71 

10. Eliminate Targets 5 17.97 2.59 .22 

11. Numerical Quotas 3 11.05 1.55 .53 

12. Remove Pride Barrier 6 27.24 4.65 .78 

13. Retraining Program 5 19.98 5.35 .87 

SUMMARY TOTALS 

Summary of above 13 Points= 68 257.72 42.90 .96 

Note: The summary total is implied to equal Deming's last point 

which is, Point 14, Take Action. 
*Cronbach, 1951. 
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TABLE #2 



INDIVIDUAL ITEM ANALYSIS INCLUDING SUBSCALE, MEANS AND 
RELIABILITY COEFFICIENTS BY SUBSCALE AND TOTAL TEST 



ITEM 


SCALE 


MEAN 


S 1 GMA 


R( TOTAL) 


R( SCALE) 


1 


g 


3 


.36 


1 . 422 


0 


. 0549 


0 


.3180 


2 


10 


'4 


.09 


1 ISA 


-0 


.0105 


0 


.4634 


3 


7 


3 


.88 


1 . O J u 


0 


. 5243 


0 


.6209 


h 


10 


2 


.88 
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